Learning Prior Feature and Attention Enhanced Image Inpainting

AbstractMany recent inpainting works have achieved impressive results by leveraging Deep Neural Networks (DNNs) to model various prior information for image restoration. Unfortunately, the performance of these methods is largely limited representation ability vanilla Convolutional (CNNs) backbones. On other hand, Vision Transformers (ViT) with self-supervised pre-training shown great potential many visual recognition and object detection tasks. A natural question whether task can be greatly benefited from ViT backbone? However, it nontrivial directly replace new backbones in networks, as an inverse problem fundamentally different To this end, paper incorporates based Masked AutoEncoder (MAE) into model, which enjoys richer informative priors enhance process. Moreover, we propose use attention MAE make learn more long-distance dependencies between masked unmasked regions. Sufficient ablations been discussed about models paper. Besides, experiments on both Places2 FFHQ demonstrate effectiveness our proposed model. Codes pre-trained are released https://github.com/ewrfcas/MAE-FAR.KeywordsImage inpaintingAttentionVision transformer